Active Length Control of Shape Memory Alloy Wires via Reinforcement Learning

The ability to actively control the shape of aerospace structures has initiated research regarding the use of Shape Memory Alloy actuators. These actuators can be used for morphing or shape change by controlling their temperature, which is effectively done by applying a voltage difference across their length. The ability to characterize this temperaturestrain relationship using Reinforcement Learning has been previously accomplished, but in order to control Shape Memory Alloy wires it is more beneficial to learn the voltage-position relationship. Numerical simulation using Reinforcement Learning has been used for determining the temperature-strain relationship for characterizing the major and minor hysteresis loops, and determining a limited control policy relating applied temperature to desired strain. Since Reinforcement Learning creates a non-parametric control policy, and there is not currently a general parametric model for this control policy, determining the voltage-position relationship for a Shape Memory Alloy is done separately. This paper extends earlier numerical simulation results and experimental results in temperature-strain space by applying a similar Reinforcement Learning algorithm to voltage-position space using an experimental hardware apparatus. Results presented in the paper show the ability to converge on a near-optimal control policy for Shape Memory Alloy length control by means of an improved Reinforcement Learning algorithm. These results demonstrate the power of Reinforcement Learning as a method of constructing a policy capable of controlling Shape Memory Alloy wire length. INTRODUCTION Advancement of aerospace structures has led to an era where researchers now look to nature for ideas that will increase performance in aerospace vehicles. The main focus of the Texas Institute for Intelligent Bio-Nano Materials and Structures for Aerospace Vehicles is to revolutionize aircraft and space systems by advancing the research and development of bioand nano-technology [1]. Birds have the natural ability to move their wings to adjust to different configurations of optimal performance. The ability for an aircraft to change its shape during flight for the purpose of optimizing its performance under different flight conditions and maneuvers would be revolutionary to the aerospace industry. To achieve the ability to morph an aircraft, exploration in the materials field has led to the idea of using Shape Memory Alloys (SMA) as actuators to drive the shape change of a wing. There are many types of SMAs which have different compositions, and in this research, nickel-titanium (NiTi) SMAs were used. SMAs have a unique ability known as the Shape Memory Effect [2]. This material can be put under a stress that leads to a plastic deformation and then fully recover to its original shape after heating it to a high temperature. This makes SMAs ideal for structures that undergo large amounts of strain, such as morphing aircraft [3]. At room temperature, SMAs begin in a crystalline structure of martensite and undergo a phase change to austenite as the alloy is heated. This phase transformation realigns the molecules so that the alloy returns to its original shape. This original shape is retained when the SMA is cooled back to a martensitic state, recovering the SMA from the strain that it had endured. When a SMA wire undergoes a crystal phase transformation, it changes its length. The phase transformation from martensite to austenite (heating) causes a decrease in length while the reverse process extends it back to its original length. Control of this transformation is needed in order for morphing actuation to be possible. The SMA wire exhibits a hysteresis behavior in its relationship between temperature and strain due to non-uniformity in the phase transformations [3]. This occurs because the phase transformation from martensite to austenite begins and ends at different temperatures than the reverse process, and the relationship is highly nonlinear. Figures 1a and 1b demonstrate this behavior, where in Figure 1a Ms is martensitic start, Mf is martensitic finish, As is austenitic start, and Af is austenitic finish.